THE EFFECTS OF RATER SEVERITY AND RATER DISTRIBUTION ON EXAMINEES' ABILITY ESTIMATION FOR CONSTRUCTED-RESPONSE ITEMS
نویسندگان
چکیده
منابع مشابه
On Rater Agreement and Rater Training
This paper first analyzed two studies on rater factors and rating criteria to raise the problem of rater agreement. After that the author reveals the causes of discrepencies in rating administration by discussing rater variability and rater bias. The author argues that rater bias can not be eliminated completely, we can only reduce the error to a cetain degree by training raters. The study on r...
متن کاملpatterns and variations in native and non-native interlanguage pragmatic rating: effects of rater training, intercultural proficiency, and self-assessment
although there are studies on pragmatic assessment, to date, literature has been almost silent about native and non-native english raters’ criteria for the assessment of efl learners’ pragmatic performance. focusing on this topic, this study pursued four purposes. the first one was to find criteria for rating the speech acts of apology and refusal in l2 by native and non-native english teachers...
15 صفحه اولA Hierarchical Rater Model for Constructed Responses, with a Signal Detection Rater Model
The hierarchical rater model (HRM) re-cognizes the hierarchical structure of data that arises when raters score constructed response items. In this approach, raters’ scores are not viewed as being direct indicators of examinee proficiency but rather as indicators of essay quality; the (latent categorical) quality of an examinee’s essay in turn serves as an indicator of the examinee’s proficienc...
متن کاملThe effect of rater severity on person ability measure: a Rasch model analysis.
This paper presents a method for analyzing oral examinations with an extended, many-faceted Rasch model that calibrates medical specialty candidates, protocols, and raters. Significant variance was found among protocol difficulties and rater severities. When candidates' raw scores were compared with calibrated measures corrected for the bias caused by the particular protocols and raters encount...
متن کاملRater Effects as a Function of Rater Training Context
This study examined the influence of rater training and scoring context on the manifestation of rater effects in a group of trained raters. 120 raters participated in the study and experienced one of three training/scoring contexts: (a) online training in a distributed scoring context, (b) online training in a regional scoring context, and (c) stand-up training in a regional context. After trai...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: ETS Research Report Series
سال: 2013
ISSN: 2330-8516
DOI: 10.1002/j.2333-8504.2013.tb02330.x